Simple screening statistical tools to detect reporting bias: when should we ask for raw data?

data raise the question of whether clinicians should trust the results published, and how evidence-based results should be translated into clinical practice [1-3]. As editors, reviewers or readers, it is our responsibility to appraise the data published or proposed for publication, before translating the results into clinical practice. Simple screening tools for detecting certain types of reporting bias would be of interest. Th e evaluation may rely on the following steps: evaluation of the distribution of the reported variables; evaluation of the distribution of the reported P-values; parametric bootstrapping and explicit computation of the P-values. In many papers, data are reported as if they were normally distributed, using summary statistics such as means and standard deviations that might not adequately refl ect the distribution of non-symmetrically distributed variables [4]. One should fi rst question whether a given variable could intrinsically behave normally or not. For instance, duration variables are usually asymmetrically distributed. Second, the reported summary statistics can provide information on the distribution. When a strictly positive variable has a standard deviation close to or even larger than its mean, the variable distribution is wide, and, if negative values are impossible, its distribution is likely to be asymmetric. Moreover, parametric statistical tests are frequently used, when they might be inappropriate if the sample size is not large enough or the distribution is too skewed. Alternatives might either rely on using non-parametric statistical tests or only on comparing the confi dence intervals without any statistical tests [5]. Statistical testing should be avoided when evaluating covariate balance since randomization should produce exchangeability. While such tests are not appropriate to accept the null hypothesis, manuscripts often report such statistical tests. Th e distribution of the corresponding P-values can then be analyzed. If randomization was adequate, baseline characteristics distribution should be balanced between groups and the P-values referring to the comparison of baseline independent characteristics should follow a uniform distribution over the interval [0,1] [6]. In case of fraud, the authors are tented to produce P-values all close to 1. Th is should probably be considered as a warning signal. However, this relies on the fact that all baseline comparisons are reported; otherwise, an adequacy test to uniform distribution could be biased because of missing-and potentially infor ma-tive-P-values. One could fi nally compute the P-values and compare them to the reported values. Th e parametric bootstrap is a simulation procedure that consists in randomly …